The present invention relates to a distributing arrangement for distribution of messages in a communication system supporting communication of packet data. The invention also relates to a communication system including a number of distributing nodes each of which includes a number of processing means. The invention also relates to a method of fault tolerant communication of messages in a communication system supporting communication of packet data. Particularly the invention relates to an arrangement and a system respectively for fault tolerant communication of messages.
In data/telecommunication systems messages are sent between different processing means. It has become common to implement LAN (Local Area Network) technology within data- and telecommunication equipment. Two reasons therefore are that the LAN technology has become attractive since it is cost effective and that the LAN technology furthermore provides a good performance. However, to provide a satisfactory operation of the systems, the communication needs to be as fault tolerant as possible to introduce reliability, redundancy and freedom from single point failures. A single LAN segment is not sufficiently reliable since the central equipment is not redundantly implemented. In known systems so called FDDI/DAS (Fiber Distributed Data Interface/Dual Attach Station) solutions or alternatively high availability ATM (Asynchronous Transfer Mode) solutions are therefore used.
Both FDDI/DAS and ATM solutions are however expensive.
An implementation of a FDDI/DAS ring is for example known. In that case redundancy is provided for through connecting each processing means to the dual ring and the handling of faults is taken care of in the points of connection.
FDDI/DAS technology is expensive and furthermore it is limited to 100 Mbps making topology restraints on nodes which require a higher bandwidth necessary. Thus, in that case the costs are still further increased since more network interfaces are needed. Furthermore the degree of freedom in software configuration is somewhat limited.
Alternatively a high availability ATM (Asynchronous Transfer Mode) switch with built-in reducancy may be used. Such a solution is however even more expensive than the FDDI/DAS solution. In some technologies the fault tolerancy is handled by protocols on a high level or on the routing level. Furthermore, through the implementation of bridges between the different networks or network segments, fault tolerant communication is not provided everywhere and the communication with some processing means may be lost if a bridge is broken.
What is needed is therefore an arrangement through which fault tolerant communication can be offered in a data/telecommunication system supporting communication of packet data. Still further an arrangement is needed through which fault tolerant communication can be provided and through which the amount of messages sent within the network, i.e. the network overhead produced hereby, can be kept low. Particularly an arrangement is needed through which it is possible to detect errors in equipment as well as on communication paths and through which it is possible to detect when a communication path etc. functions satisfactorily again and through which messages can be sent from one processing means to another with a high degree of availability, i.e. with a high probability of success. Still further an arrangement is needed through which faults can be detected in a fast and efficient manner thus enabling restoration of errors.
Particularly an arrangement is needed through which the provision of fault tolerance is transparent for high level protocols and on the routing level. Still further an arrangement is needed which allows the use of standard components (hardware). Particularly an arrangement is needed which is cheap and which has a high performance and which is reliable, shows a high degree of redundancy and which provides freedom from single point failures. Still further an arrangement is needed which enables an automatic error detection and failover as well as automatic recovery. Particularly an arrangement is needed which enables operator notification of failures particularly while the operation still is substantially unaffected.
Particularly a distributing arrangement for distribution of messages in a communication system is needed which meets the above mentioned requirements. Still further a communication system supporting communication of packet data is needed through which the above mentioned objects are achieved.
Further yet a method providing fault tolerant communication in a communication system supporting communication of packet data is needed through which the above mentioned objects are achieved.
Therefore a distributing arrangement for distribution of messages in a communication system is provided which comprises a number of processing means which communicate with each other through sending of messages. Each processing means is connected to all the other processing means via at least two communication networks or network segments which are physically separated from each other. For each processing means information holding means are provided for keeping information about all communication paths to all other processing means and this information is obtainable by way of sending first messages, particularly with a given frequency, from each processing means to all the other processing means using all of said at least two communication networks. Such first messages may be sent separately but they may also be piggy-backed on e.g. ordinary traffic messages, or a combination of both. Advantage may also be taken of information indirectly obtainable through the ordinary traffic that flows.
In an exemplary embodiment a predetermined time interval is given for each processing means and first information messages sent from one processing means substantially simultaneously to another processing means using at least two physically separated communication paths including different networks should arrive within the predetermined time interval. If they do so, the at least two different communication paths using different communication networks both function. In an advantageous embodiment the first information messages are special purpose broadcast messages and preferably each processing means send their respective special purpose broadcast messages on all networks or network segments substantially simultaneously. Alternatively the first messages consist of a burst of unicast messages. Particularly each processing means comprises means for detecting anomalies or asymmetries in the reception of such first messages from one and the same sending processing means and when such an asymmetry or anomaly is detected, the receiving processing means enters a decision process to localize the error and or to establish the kind of the error. In a particular embodiment means are provided for comparing the reception of first messages during a first time interval and the reception of first messages during a subsequent observation time intervals. If an anomaly or an asymmetry in the reception during the first observation time interval still remains in the subsequent observation time interval, the receiving processing means sends a second message, e.g. a unicast message containing information about an error or about the result of the sending of the first messages, to the sending processing means on all communication networks/network segments. In an alternative embodiment a second message, e.g. a unicast message, is sent to the sending processing means on all networks/network segments as soon as an error is detected. Still further, more than two observation periods during which one and the same anomaly remains could be the requirement for sending out a second, e.g. a unicast message containing an error announcement. With an asymmetry or anomaly is meant that not all first messages from one and the same sending processing means arrive within the time window of the receiving processing means. It is supposed that each processing means knows which first messages that should be received. Actually it is sufficient to know the number of networks.
For each processing means there may of course be several parallell fault detection processes run at the same time towards different other processing means due to working within the time window and since all processing means communicate with all other processing means.
Particularly, if no messages at all are received in a receiving processing means over one of the networks, said processing means provides for transmission of a broadcast message which is broadcasted to all other processing means using said at least two networks (all available networks) to inform said other processing means that no messages whatsoever are received on that particular network.
In a particular embodiment each processing means comprises means for generation of an alarm message if an anomaly or an asymmetry remains for a given number of observation time periods, which alarm message then is sent to an operation system managing the distributing arrangement using all (available) of said at least two networks.
Instead of using a time window to detect anomalies the messages can be numbered in an appropriate manner, e.g. in sequence, and then can be detected when a message numbered in a particular way is only received e.g. over one network or if it is completely missing etc. since a packet numbered in the same way should be received on all interfaces.
Particularly, in each processing means a table is dynamically built using received unicast messages and possibly information in received broadcast messages (if the network(s) support(s) a broadcasting functionality), which information is used to control the sending of messages from the respective processing means. In a particular embodiment each processing means keeps information about the existence of all other processing means. Particularly said tables contain information about all usable communication paths to all peers, i.e. all other processing means, with which the respective processing means communicate.
In an alternative embodiment each processing means comprises means for sending response messages with information about received first messages, e.g. special purpose broadcast messages, at least to the respective sending processing means. Particularly the information about received first messages is provided in unicast special purpose messages which are sent to the respective sending processing means and they include information about on which network(s) first message(s) is/are received. Said response unicast special purpose messages are sent on all (available) of said at least two networks substantially simultaneously. In a particular embodiment said response messages are used to dynamically build a table in information holding means provided for each processing means which information is used to control the sending of messages from the respective processing means. Particularly said table shows all usable communication paths to all peers, i.e. all processing means, with which a respective processing means communicate.
In particular embodiments an intermediate message handling device is connected to each processing means (or some of the processing means) respectively for handling the sending of said first messages and second messages. In the embodiment implementing sending of response messages, said intermediate devices handle the first messages and response messages.
In a particular embodiment an intermediate message handling arrangement is provided containing all (or a number of) the intermediate message handling devices. In an alternative embodiment means are provided in each (or some of the) processing means for handling the generation/sending/receiving of first and second messages or first messages and response messages.
In a particular implementation the information provided in the information holding means is also used for controlling the sending of messages so as to share the load between the communication paths in an appropriate manner.
In one embodiment the at least two networks comprise separate networks such as different local area networks or networks based on different technologies. There may be more than two networks. Of course also networks based on one and the same technology may be used. In a particular advantageous embodiment the networks support a broadcasting functionality. Alternatively some network(s) support(s) a broadcasting functionality whereas one or more other network(s) do not.
Therefore also a communication system supporting packet data communication is provided which includes a number of distributing nodes for distribution of messages. Each node comprises a number of processing means communicating with each other. All processing means communicate via at least two different networks or network segments which networks/network segments are physically separated. Each processing means comprises or is associated with means for sending first information messages to all other processing means which messages are sent on each of said at least two networks. For each proccessing means information holding means are provided in which the information resulting from the sending of said first information messages is collected to provide information about the available communication paths to all other processing means respectively.
Examples on nodes in which the inventive concept can be implemented are the SGSN (Serving GPRS Support Node) and the GGSN (Gateway GPRS Support Node) wherein GPRS (General Packet Radio Service) is the packet data service of GSM, c.f. e.g. GSM 03.60 V5, 2.0, 1997-12 and the P-MSC of PPDC which is the packet data service of PDC (Pacific Digital Communications).
In a particularly advantageous embodiment the first information messages consist of special purpose broadcast messages and in each processing means detecting means are provided for detecting if first information messages sent from particular sending processing means using the at least two separate networks arrive within a predetermined time interval as well as means of sending a special purpose unicast message to a sending processing means in case messages are not received on all networks within the predetermined time interval for a given number of observation time intervals, to indicate the result or that one of the communication paths contains an error. Particularly, if a processing means does not receive any message on a particular network at all, information thereon is provided in its own next first information broadcast message which is provided to all other processing means using all available networks.
Alternatively the first messages are a burst of unicast messages, e.g. in case a network does not support broadcasting. In an alternative embodiment each processing means comprises means for sending response messages to sending processing means upon reception of first information messages from said sending processing means. Particularly said response messages comprise unicast response messages including information about over which network(s) first information messages is/are received. Particularly each processing means comprises or is connected to information holding means in which a table is built using the information received in second information messages (unicast special purpose messages) or response messages so as to dynamically provide for updated information about available communication paths to all other processing means.
In a particular embodiment message handling means (intermediate devices) responsible for the sending/receiving of first messages and second messages or response messages are provided in connection to one or more (or all) processing means and form an intermediate arrangement comprising one intermediate device connected to each processing means. In an alternative embodiment the message handling means for handling the sending/receiving and generation of first messages and second messages or response message are comprised in the respective processing means.
The at least two networks may be networks of the same kind, or of different kinds and there may be two, three or more different networks. Examples on networks are LAN (Local Area Network), Ethernet, FDDI and ATM. Advantageously the networks support broadcasting functionality.
Therefore also method of providing fault tolerant communication in a communication system supporting communication of packet data and comprising a number of distributing nodes, each of which comprises a number of processing means, is provided. The method comprises the steps of; interconnecting each processing means with all other processing means via at least two physically separated networks; sending first information messages from each processing means to all the other processing means with a given frequency (at least if needed; information obtainable through ordinary traffic may also be taken advantage of) using all of said at least two networks; providing information to at least the sending processing means about faults concerning first information messages sent from the respective sending processing means to the respective receiving processing means on the at least two networks substantially simultaneously; collecting information obtainable through the sending of said first messages for each of said processing means in information holding means using said information to detect errors in the communication paths; and using the information to control the communication between the respective processing means and to keep information about errors.
In a preferred implementation the first information messages comprise special purpose broadcast messages and the method further includes the steps of, when detecting an error, sending a second message comprising a special purpose unicast message to the sending processing means using all of said at least two networks. Alternatively the first messages are a burst of unicast messages.
Particularly the method comprises the steps of defining a time window for each processing means and if all first information messages from one and the same sending processing means using different networks do not arrive in a receiving processing means within the time window; informing the sending processing means. In a particular embodiment the method comprises the steps of, if an error is detected in the communication path when a particular network is used for sending of a message from a particular sending processing means, sending a unicast message to the sending processing means and if it is detected that there is a malfunctioning in the receiving processing means or an error on one of the networks, including information about that in the next first message, e.g. special purpose broadcast message of the receiving processing means which then is broadcasted using all the networks. In an alternative embodiment the method comprises the step of confirming reception of first messages in response, e.g. unicast messages sent to the respective sending processing means, using each network, to provide information about usable communication paths on every network and building a table with information about available communication paths using said response messages in each processing means.
According to different embodiments, as referred to above, the second messages or the response messages resulting from the sending of first messages are used to dynamically build an information table in the respective processing means. It is an advantage of the invention that a high availability on system level provided and that fault tolerant communication is provided in a way that is transparent to the users of the data link services. It is also an advantage that no single point of failures are provided and also that automatic error detection is provided for and that an automatic recovery is enabled. It is also an advantage that it is possible to handle dual unit transmitter/receiver errors even if they are located on separate networks as long as there exists at least one usable transmission path in each direction. According to the invention it is particularly possible to remove one whole network, e.g. a whole local area network, such as for example a switch, in operation without requiring any particular actions by the operator.